Operator level microbenchmarking #3154
Conversation
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D77676673 |
Summary: This change introduces microbenchmarking for PyTorch operators. Since we need to capture and measure each operator call (which is happening under the hood of PyTorch), we need to use torch.profiler.profile. Example operators are `aten:mm`, `aten::sigmoid`, `cudaLaunchKernel`, etc… Use `--benchmark_operators` to enable the operator-level benchmarking. Use `--limit_operator_results` argument to specify the number of top runtime operators to benchmark. Use `--target_operators` argument to list PyTorch operators to benchmark. Example output: ``` TrainPipelineSparseDist | Malloc retries (P50/P90/P100): 0.0 / 0.0 / 0.0 | Runtime (P90): 442.08 ms | Peak Memory alloc (P90): 24.23 GB | Peak Memory reserved (P90): 26.21 GB operator_aten::copy_ | Malloc retries (P50/P90/P100): -1.0 / -1.0 / -1.0 | Runtime (P90): 39.21 ms | Peak Memory alloc (P90): 0.00 GB | Peak Memory reserved (P90): -0.00 GB ... ``` Differential Revision: D77676673
SSYernar
force-pushed
the
export-D77676673
branch
from
da06c9d to
f67ee85
Compare
Summary: Pull Request resolved: pytorch#3154 This change introduces microbenchmarking for PyTorch operators. Since we need to capture and measure each operator call (which is happening under the hood of PyTorch), we need to use torch.profiler.profile. Example operators are `aten:mm`, `aten::sigmoid`, `cudaLaunchKernel`, etc… Use `--benchmark_operators` to enable the operator-level benchmarking. Use `--limit_operator_results` argument to specify the number of top runtime operators to benchmark. Use `--target_operators` argument to list PyTorch operators to benchmark. Example output: ``` TrainPipelineSparseDist | Malloc retries (P50/P90/P100): 0.0 / 0.0 / 0.0 | Runtime (P90): 442.08 ms | Peak Memory alloc (P90): 24.23 GB | Peak Memory reserved (P90): 26.21 GB operator_aten::copy_ | Malloc retries (P50/P90/P100): -1.0 / -1.0 / -1.0 | Runtime (P90): 39.21 ms | Peak Memory alloc (P90): 0.00 GB | Peak Memory reserved (P90): -0.00 GB ... ``` Differential Revision: D77676673
|
This pull request was exported from Phabricator. Differential Revision: D77676673 |
SSYernar
force-pushed
the
export-D77676673
branch
from
f67ee85 to
484444e
Compare
|
Hi @SSYernar! Thank you for your pull request. We require contributors to sign our Contributor License Agreement, and yours needs attention. You currently have a record in our system, but the CLA is no longer valid, and will need to be resubmitted. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at [email protected]. Thanks! |
Summary:
This change introduces microbenchmarking for PyTorch operators.
Since we need to capture and measure each operator call (which is happening under the hood of PyTorch), we need to use torch.profiler.profile. Example operators are
aten:mm,aten::sigmoid,cudaLaunchKernel, etc…Use
--benchmark_operatorsto enable the operator-level benchmarking.Use
--limit_operator_resultsargument to specify the number of top runtime operators to benchmark.Use
--target_operatorsargument to list PyTorch operators to benchmark.Example output:
Differential Revision: D77676673